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1. Introduction 

The setup. Considered are perfect information games played over a game tree of infinite depth. A single active 
player is assigned to every history in the tree. The play begins at the empty history, called the root of the 
tree. Once a given history is reached, the corresponding active player chooses an action leading to one of the 
successor histories. This generates an infinite sequence of actions, called a play, which determines the payoffs. 

We allow for an arbitrary set of players. Without loss of generality we require that each history has at least one 
successor, and we impose no further restrictions on the game tree. We assume that each player’s payoff function is 
bounded and Borel measurable with respect to the topology generated by the cylinder sets, as e.g., in Martin [20]. 
This is a very mild assumption. Games with semicontinuous payoffs (Purves and Sudderth [24], Flesch et al. [7]), 
perfect information stopping games (e.g., Solan [25], Mashiah-Yaakovi [21]), recursive stochastic games with perfect 
information (e.g., Flesch et al. [8], Kuipers et al. [16]) are all examples of games with Borel measurable payoffs. 

Parts of the paper are written under an additional assumption that each player’s payoff function only takes 
finitely many distinct values. The main reason to focus on payoff functions with finite range is that they serve 
as natural approximations of bounded payoff functions. 

This paper concentrates on the concept of subgame-perfect (e--)equilibrium in pure strategies. The main 
contribution of the paper is twofold. We discuss each contribution, and the additional results, in turn. 

Contribution I. Our first contribution is a characterization of subgame-perfect equilibrium plays, in games 
where the payoff functions are Borel measurable and have finite range. The stated characterization is obtained 
using an algorithm of iterative elimination of plays. 

The algorithm proceeds as follows. At each step of the algorithm and for each history of the game tree we 
determine a certain payoff level, called the security level, for the player who is active at the given history. This 
is the highest payoff level that the active player can be sure to attain when playing against any strategy profile 
that only prescribes plays that have survived the elimination thus far. We then eliminate all plays with a payoff 
lower than the active player’s security level. The algorithm terminates when no more plays can be eliminated. 
The plays that remain are exactly those that are supported by some subgame-perfect equilibrium of the game. If 
no plays remain, the game admits no subgame-perfect equilibrium. In general, the algorithm takes a transfinite 
number of steps to terminate. 

The main technical novelty of our work is in the definition of the security payoff level. The definition relies 
on an auxiliary zero-sum game played by the deviator (Player I) against the strategy builder (Player II). Player I 
effectively represents the active player at the given history in the original game, while Player II acts as his 
artificial adversary. 


1162 


Flesch and Predtetchinski: Subgame-Perfect Equilibrium Plays in Borel Games of Perfect Information 
Mathematics of Operations Research, 2017, vol. 42, no. 4, pp. 1162-1179, ©2017 INFORMS 1163 


The game begins with Player I choosing an action. Player II then proposes a play that respects Player I’s 
move. Player I then chooses to accept the proposed play or to deviate. In the former case, the game ends. In the 
latter case, Player I also announces a deviation, and Player II proposes a new play that accommodates Player I’s 
deviation. This process continues ad infinitum. Player II is restricted to propose plays that have survived the 
elimination thus far. The value of this auxiliary zero-sum game is defined to be the active player’s security level. 

We believe that the characterization established here could yield useful insights into the nature of subgame 
perfection. Our second contribution on upper semicontinuous games, obtained as an application of this charac- 
terization, serves as an illustration. 

Contribution II. As an application of the characterization result, we address an open question raised by Purves 
and Sudderth [24]. Purves and Sudderth work ina setup that is identical to ours, except for their assumption that the 
number of players is finite. They show that if each player’s payoff function is bounded and upper semicontinuous, 
then for each positive e, the game admits a subgame-perfect €-equilibrium. The authors further pose the question 
of whether the assumption that the number of players be finite could be dispensed with. 

The technique that Purves and Sudderth use is an induction on the number of distinct payoff vectors in the 
game. Because of this feature of their proof, extending it beyond the case of finitely many players seems to be 
difficult. We take a different approach. We show that in games where each player’s payoff function has a finite 
range and is upper semicontinuous, our algorithm returns a nonempty set of plays at each step of the iteration. 
This allows us to deduce that any such game has a subgame-perfect equilibrium. The result for games with 
bounded upper semicontinuous payoff functions then follows by discretizing the payoff functions, exactly as is 
done by Purves and Sudderth. 

Additional results. For games with bounded Borel measurable payoffs we obtain the following result: 
subgame-perfect equilibrium plays survive all stages of the elimination. Conversely, a play that survives all 
stages of elimination could be supported as a subgame-perfect €-equilibrium play for each positive e. We show 
by means of an example that this result cannot be strengthened: there might be plays that survive all stages 
of elimination and yet that are not supportable by any subgame-perfect equilibrium. We argue that such a 
discrepancy arises because Player II might not have an optimal strategy in the auxiliary zero-sum game. 

Our final result is motivated by the frequent use of the technique of discretizing payoff functions: approxi- 
mating a given bounded payoff function by a function with finite range, as is done, for instance, by Mertens 
and Neyman (see Mertens [22]), Flesch et al. [7], Purves and Sudderth [24], Flesch and Predtetchinski [10, 11]. 
The technique allows one to reduce the problem of existence of (subgame-perfect) e-equilibrium for a game 
with bounded payoff functions to the problem of existence of (subgame-perfect) 0-equilibrium for a game where 
the payoff functions have finite range. We show that a game with bounded Borel measurable payoff functions 
admits a subgame-perfect €-equilibrium for each e€ > 0 if and only if, for each e€ > 0, there exists a Borel measur- 
able e-discretization of its payoff functions that admits a subgame-perfect 0-equilibrium. Our result could be 
interpreted as saying that passing to discretizations is without loss of generality, at least if one is only interested 
in the question of existence of subgame-perfect e-equilibrium for every positive e. 

Related literature. Closely related to the algorithm developed here are recursive algorithms in Harris [14] 
and in Flesch et al. [7]. Both algorithms terminate with the set of subgame-perfect equilibrium plays. Harris 
considers the class of continuous games, whereas Flesch et al. assume that the payoff functions are lower 
semicontinuous. For general payoff functions having finite range, the algorithm in Flesch et al. outputs the set 
of plays that could be supported by a one-deviation immune strategy profile, that is, a strategy profile such 
that no player could improve his payoff by deviating at a single history only. 

Our algorithm shares a number of features with those in Harris and Flesch et al.: all three algorithms prescribe 
iterative elimination of plays. At each step of the algorithm one determines a certain payoff level that a player 
could, in some sense, guarantee for himself. What distinguishes the algorithm presented here is the way the 
security payoff level is determined. 

Harris and Flesch et al. only take into account deviations at a single history. More precisely, each of the 
player’s actions is evaluated, assuming that the ensuing play will be the worst possible among those plays that 
survived the elimination thus far. Equivalently, one could see this as a value of a “short” zero-sum game, where 
the deviator is only allowed to deviate once, and is forced to accept the first proposal of the strategy builder. 
In contrast, the “full” auxiliary zero-sum game employed here allows the deviator to deviate finitely or even 
infinitely many times. Unlike the algorithms of Harris and Flesch et al., the algorithm proposed here does not 
require the game to satisfy the one-shot deviation principle: indeed, the security payoffs capture the possibility 
of infinite as well as finite deviations. 

Flesch et al. [12] consider the class of free transition games and proves the existence of subgame-perfect 
e-equilibrium in mixed strategies using an algorithm of iterative elimination of plays. This algorithm operates 
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with pairs of plays, rather than individual plays. In the construction of subgame-perfect e-equilibria, the primary 
play is to be followed with a high probability, and the secondary play, which acts as a threat, is to be followed 
with a small probability. Kuipers et al. [16] refine this algorithm to establish the result for perfect information 
stochastic games with the crucial feature that every player controls only one state. Alds-Ferrer and Ritzberger [1] 
consider an algorithm very different from ours to obtain an existence of subgame-perfect equilibrium in a class 
of continuous games. 

Fudenberg and Levine [13] and Harris [15] develop a characterization of subgame-perfect equilibrium strate- 
gies (rather than plays) by means of convergent sequences of subgame-perfect e-equilibria in the truncated 
games. Carmona [5] provides a characterization in terms of approximations of the game by payoff functions 
with finite range. Alés-Ferrer and Ritzberger [2] derive a condition on the topology of the set of plays that is 
both necessary and sufficient for the existence of a subgame-perfect equilibrium. 

Brihaye et al. [4] obtain a characterization of one-deviation immune strategy profiles (called very weak 
subgame-perfect equilibria there) for quantitative games played on finite graphs, using an algorithm motivated 
by the one in Flesch et al. (2010a). 

Le Roux [18] provides an intriguing analysis of games with A—payoff functions. Essentially, he shows that, 
in the two-player case, the game by Solan and Vieille [26] is the only payoff pattern that could rule out the 
existence of subgame-perfect equilibrium. 

Le Roux and Pauly [19] derive a number of existence results for subgame-perfect €-equilibrium with infinitely 
many players. These results are transfer results, i.e., starting with the existence of some type of equilibria for a 
small class of games, they allow one to conclude the existence of some type of equilibria for a larger class. Our 
result on discretization could also be seen as such a transfer result. 

T. Brihaye, V. Bruyére, N. Meunier, and J.-F. Raskin (reported in Meunier [23]) provide a characterization of 
games that admit a subgame-perfect equilibrium. With each perfect information game, they associate a two- 
player zero-sum game, called the prover game. They show that the original perfect information game has a 
subgame-perfect equilibrium if and only if player I has a winning strategy in the corresponding prover game. 
They also give a similar characterization for one-deviation immune strategy profiles (called very weak subgame- 
perfect equilibria there). 

As we allow the player set to be infinite, our setup encompasses some classes of intergenerational games: 
perfect information games where a player represents a generation and thus acts exactly once in the course of 
the game. For a thorough review of such games, we refer to Balbus et al. [3]. In a related work, Cingiz et al. [6] 
consider infinite centipede games where each player acts exactly once. Using truncations of the game tree, the 
authors show the existence of subgame-perfect equilibrium under the assumption that the payoff functions are 
upper semicontinuous. 

The organization of the paper. In Section 2 we describe the class of perfect information games. In Section 3 
we provide an informal overview of the algorithm, describe a formal definition, and discuss two illustrating 
examples. Section 4 provides an overview of the main results. Proofs of the main results are carried out in 
Sections 5 and 6. Section 7 is dedicated to the discussion of interesting special classes of games. Section 8 
contains additional results. 


2. The Model and Preliminaries 

Finite and infinite sequences. Let N = {0,1,2,...} and let A be a nonempty set. We denote by A®™ the set of 
all finite sequences of elements of A, including the empty sequence Ø. A finite sequence of length t + 1, where 
t €N, is denoted by (ao, ...,a,). However, with a slight abuse of notation, we write a to denote the sequence (a) 
of length 1. For h € H we write (h,a) to denote the concatenation of h by a. We let A denote the set of all 
infinite sequences of elements of A. A typical element of A is denoted p = (agọ, 44,- .. ). 

For a finite sequence h = (a,...,a,) € AN, sequences Ø, (ao), (A0, 41), -- -, (A07 - --,4,), are called prefixes of h. By 
definition, the only prefix of @ is Ø. We write g < h to mean that g is a prefix of a finite sequence h, and we 
write g < h if g is a prefix of h and g +h. For an infinite sequence p = (ag, 44, ...) E€ AN, finite sequences Ø, (4), 
(ao, 41), (40,41,42), ..., are called prefixes of p. We write h <p to mean that h is a prefix of p. 

Given h,e € A®™ and p € A we let (h,e) and (h, p) denote concatenations defined in an obvious way. 

The game Q. Let A be an arbitrary nonempty set. Elements of A are called actions. Let H be a subset of A^ 
such that (1) @ € H, (2) if h € H, then every prefix of h is an element of H, and (3) if h € H, then there is a € A 
such that (h,a) € H. Elements of H are called histories. A play p is an element of AN such that every prefix of p 
is an element of H. The set of all plays is denoted’ by P. 

Let I denote an arbitrary nonempty set of players. Let ı: H — I be a function that assigns an active player 
to each history. Further, each player i € I is given a so-called payoff function u;: P > R. Thus a game Q is 
characterized by the sets I, A, H, and the functions 1 and u; for each i € I. 
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The game Q is played as follows: let họ = Ø. Suppose a history h, € H has been determined for some t €N. 
Player 1(h,) then chooses an action a, such that (h,,a,) € H. The players thus generate a play p = (ao, 41,- . .), and 
each player i € I receives payoff u;(p). 

This setup encompasses all games of finite duration. Another important special case is the situation when the 
players receive instantaneous payoffs at every period of the game and then aggregate them into one payoff, for 
example, by taking the discounted sum. 

The topological structure. We endow the set A with the discrete topology, the set AN with the product 
topology, and the set P with the topology as a subspace of A. Denote this topology by J. A basis for (P, F) is 
formed by the cylinder sets P(h) = {p € P: h < p} for he H. 

The space (P,J) is completely metrizable. One can take, for example, the metric d: P x P > R given by 
d(p,q)=2*” if p q, where k(p,q) €N is the length of the longest common prefix of p and g. A sequence 
of plays (p,,) nen converges to a play p precisely when for every k € N there exists an N, € N such that for every 
n > N, the plays p„ and p have a common prefix of length k. The space (P, 7) is separable (and hence Polish) if 
and only if H is a countable set.* This is clearly equivalent to the condition that for every h € H there are only 
countably many a € A such that (h,a) € H. 

A function f: P — R is said to be upper semicontinuous if for each x €R, the set {p € P: f(p) 2 x} is closed. It is 
easy to see that the following conditions are equivalent: (A) f is upper semicontinuous, (B) for every sequence 
of plays (P,,) nen Converging to a play p, we have limsup,,_,., f(p,,) < f(p), and (C) for every p € P and every 6 >0 
there is a prefix h of p such that f(p) > f(q)— ô for every q € P(h). Intuitively, condition (C) means that any 
play p has a prefix h such that p is maximal, or at least almost maximal, within P(h). 

A function f: P >R is said to be lower semicontinuous if the function —f is upper semicontinuous. A function f 
is continuous if and only if it is both upper and lower semicontinuous. 

Strategies. In this paper, we only consider pure strategies, unless specified otherwise. A (pure) strategy for 
player i is a function g; assigning to each history h € H with (h) =i an element g;(h) of A such that (h, o;(h)) € H. 
The interpretation is that if a history h as above arises, then g; tells player i to choose the action g;(h). A strategy 
profile is a tuple o =(0;);-; where o; is a strategy for player i. Given a strategy profile ø and a strategy n; for 
player i, we write o/1; to denote the strategy profile obtained from o by replacing o; with n;. 

A strategy profile ø induces a unique play at a history h € H, denoted m(o,h). It is an element of P(h). In 
particular, the induced play at the root is n(o, Ø). 

Subgame-perfect e-equilibrium. Let e€ > 0 be an error term. A strategy profile o is called an e€-equilibrium if 
no player can gain more than e by a unilateral deviation, i.e., if for each player i € I and for each strategy o; of 
player i it holds that 

u;(n(o, @)) = u;(m(a/0;,@)) —e€. 


Mertens and Neyman (see Mertens [22]) showed that an e-equilibrium exists for every e > 0, provided that 
the payoff functions are bounded and Borel measurable. Their proof is for finitely many players, but it can be 
extended to infinitely many players along the same lines (see also Le Roux and Pauly [19]). 

Subgame-perfect e-equilibrium is a refinement of e-equilibrium. A strategy profile o is called a subgame-perfect 
e-equilibrium, or e-SPE for brevity, if no player can gain more than e by a unilateral deviation conditional on 
any history. Formally o is an e-SPE if for each history h € H, each player i €I, and each strategy o; of player i 
it holds that 

u,(m(o,h)) > u,(m(o/0;,h)) —e. 


A 0-SPE is simply called an SPE. For every h € H define 
P*(h) = {n(0,h): o is an SPE in Q}. (1) 


The set P*(h) consists of all plays that are induced by an SPE of the game Q at history h. The set P*(h) is a 
subset of P(h). We refer to elements of the set P*(@) as SPE plays. 

Flesch et al. [9] provided a two-player game in which each player’s payoff function is bounded and Borel 
measurable, yet the game admits no e-SPE for small e > 0, not even in randomized strategies. 

Zero-sum games. We make use of the following lemma due to Mertens and Neyman (see Mertens [22], 
p. 1567). The result ultimately rests on Borel determinacy by Martin [20]. 


Lemma 2.1. Let Q be a two-player zero-sum perfect information game having a bounded and Borel measurable payoff 
function. Then the game Q has a value. Moreover, for each e > 0, each player has a strategy that is e-optimal in every 
subgame (and, consequently, the game has an €-SPE). If, in addition, the payoff function has finite range, then each player 
has a strategy that is optimal in every subgame (and, consequently, the game Q has an SPE). 
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3. The Algorithm 
In this section we discuss the algorithm of iterative elimination of plays that outputs the collection of sets P*(h) 
for h € H. We split this section into three sections. In Section 3.1, which is not intended to be rigorous, we give 
a nontechnical overview and a motivation for the algorithm. In Section 3.2 we give a formal definition of the 
algorithm. In Section 3.3 we illustrate the algorithm by means of two examples. 

Throughout this section, we assume that the payoff function of each player has finite range and is Borel 
measurable. 


3.1. An Overview 

We recursively define, for each history h, the sequence P;(i) of sets, and sequences f:(h) and a;(h) of payoffs, 
indexed by an ordinal number €. The set P;(h) contains the plays that have survived the elimination up to 
step € of the algorithm. Naturally, the sequence P;(h) is nested: as the algorithm proceeds, fewer and fewer 
plays remain. The number a;(h) is the security level for the player active at the history h. This is the payoff 
level that the active player can be sure to guarantee for himself when playing against any strategy profile that 
only prescribes plays that have survived the elimination process up to stage €. The number f;(h), which is only 
interesting when € is a limit ordinal, is the maximum of the security payoff levels a,(h) over the ordinals A 
smaller than č. 

Initialization. We initialize the recursion by letting P(A) be the set of all plays having h as a prefix, and letting 
Bo(h) = ay(h) be the worst payoff for the player (h) in the subgame starting at the history h. 

Step 1. At step 1 of the algorithm, we set $4 (h) = a)(h) and P,(h) = Po(h), and we let the security level a,(h) 
be the value of the following zero-sum game, denoted G,(/). 

Player I of the game G,(h) is called the deviator, and Player II is called the strategy builder. Player I effectively 
represents player (h) in the original game, while Player II acts as his artificial opponent. The game starts with 
Player I taking an action ay at the history họ =h. Player II responds by proposing a play pọ extending the prefix 
(ho, a9). Player I now has a choice of either to accept pọ, or to deviate. If Player I accepts, the game ends. If 
Player I chooses to deviate, he announces a prefix h, of po extending (hy,4)) such that (h) = (họ), and an 
action a, that is incompatible with pọ. Player II responds by proposing a new play, p,, that extends the history 
(h,,4,). This process continues ad infinitum. In this way, the players generate the outcome play q: if Player I 
accepts Player II’s proposal p,, then q = p,, while if Player I never accepts any proposal, then q is defined to be 
the play extending the histories họ, h4, .... Player I’s payoff is given by u,;,)(q). 

Thus a,(h) is the highest payoff player ı(h) can be sure to attain when playing against an arbitrary strategy 
profile of his opponents. It is clear that in an SPE, player 1(/)’s payoff cannot be smaller than a,(/). Indeed, if it 
were smaller than a,(/), player ı(h) could “beat” the assumed equilibrium and gain at least a,(h), by adopting 
Player I’s optimal strategy in the game G,(i). This means that, at the next step of the algorithm, we can safely 
eliminate any play p having h as a prefix that gives player ı(h) a payoff of less than a,(h). 

Step 2. At the second step of the iteration, we let $ (h) = a,(h), and we let P,(h) consist of plays p having h 
as a prefix, such that for each prefix g of p extending h, player 1(g)’s payoff on p is at least 6,(g). In view 
of the above discussion, P,(h) includes P*(h). The security payoff level a,(h) is defined to be the value of the 
game G,(i). The rules of the game G,(h) are similar to those of G,(h), but Player II is now restricted to propose 
a play that is an element of the set P,(h,,a,), after a move (h,,a,) of Player I. 

Thus a,(h) is the highest payoff level player 1(h) could be sure to get against any strategy profile that only 
prescribes plays that survived the first round of elimination. It is also clear that player :(h)’s payoff at history h 
in any SPE is not smaller than a,(h). 

General step. At a general step € of the iteration, we let 6;() denote the highest security level a,(/) over all 
iterations A < €, and we define P;(h) as the set consisting of all plays p extending h that give a payoff of at 
least f(g) for the player active at a prefix g of p. By a reasoning similar to that above we can see that P;(h) 
includes P*(h). The security level a;(h) is defined as the value of the game G;(h) where Player II is restricted 
to propose plays from the set P-(h;,4a,). 

Limit properties. For every history h the sequences thus defined enjoy the obvious monotonicity properties: 
the sequence P;(h) is nonincreasing by inclusion, and the sequences a;(i) and f;(/) are nondecreasing. For a 
sufficiently large ordinal number ¢*, we have P,.(h) = P*(h) for every history h € H. Whenever Q has an SPE, the 
sets P*(h) are nonempty. In this case a+ (h) = B;.(h) is the minimum of 1(h)’s payoff function on the set P*(h). 


This algorithm shares a number of features with that in Flesch et al. [7]. In particular, it produces a nested 
sequence of sets of plays P(A), and a nondecreasing sequence of security payoff levels a;(h). The novelty of 
our approach lies in the way the security payoff levels a;(h) are determined. 
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The definition of the payoff levels a;(h) in Flesch et al. could be interpreted as being the value of the “short” 
auxiliary zero-sum game in which Player I chooses action ag, Player II chooses a play pọ extending the history 
(hy, 49), and Player I has no choice but to accept pọ. In contrast, in the “full” zero-sum game used here, Player I 
is allowed to deviate arbitrarily often from the proposed plays. 

This explains why our algorithm enjoys very different limit properties than the algorithm in Flesch et al. 
The limit of the algorithm in Flesch et al. is the set of plays that could be supported by strategy profiles that 
are immune to one-shot deviations. It is typically a superset of plays that could be supported by an SPE. Our 
algorithm outputs the latter set. The two sets coincide if the game satisfies the one-shot deviation principle. For 
a more elaborate discussion of one-deviation immunity we refer to Section 7. 

Similar remarks apply to the comparison between our algorithm and that in Harris [14]. The essential differ- 
ence (apart from the obvious differences in the setup) is the definition of the security levels. 


3.2. The Formal Definition 
For the ordinal 0 we define 
Po(h) = P(h), 
alh) = polh) = BER Un (P)- 


Let ®(0) denote the statement that P)(/) is nonempty for each h € H. The statement ®(0) is obviously true. 
For each ordinal € > 0 define 


Be(h) = max a(h), 2 
for each g € H a 
P:(h)= 4p € P(h): : 3 
af ) [p (h) uxa (p) > B:(8) (3) 
We also define ®(¢) to be the following statement: 
“P (h) #@ for each A< € and hE H.” (4) 


If Ẹ(&) is false, the algorithm terminates. In that case, for the sake of completeness we adopt the convention 
that P,(4) =Ø for each A > € and each history h € H. 

Assuming that the statement ®(€) is true, consider an auxiliary perfect information game G;(h). This is a 
zero-sum game where Player’ I essentially represents player 1(). Players I and II move alternately, starting 
with Player I, who makes an initial move of the form (hy,4)) € H, where họ = h. Suppose that, at period k of 
the game, Player I chooses a move (h,a) € H. Then, Player II chooses a play p € P:(hy,a,). Subsequently, at 
period k +1, Player I chooses either to accept p;,, or to make a move (hj41, 4441) € H, where (1) (hy, ap) < Nyy < Pk 
and (h4) = uh), and (2) the sequence (hy,1,4;4;) is not a prefix of p,. In the latter case we say that Player I 
deviates from p, at the prefix h,,, with the action a,,,. If there is no move (h;,,,4,,,) with the stated properties, 
Player I has no choice but to accept p,. The outcome of the game is the unique play q extending the histories 
h=h) <h,<--- if Player I never accepts, or the play q =p, if Player I accepts p, at period k +1. Player I’s payoff 
is then u, (q), which is to be paid by Player II. 

Finally, we define the security level as 

a;(h) = val(G;(h)), 
the value of the game G,(h). The existence of the value is shown below in Lemma 3.1. This completes the 
recursive definition. l 


Lemma 3.1. Let € be an ordinal such that the statement ®(&) is true. Let h € H. The payoff function of the game G;(h) is 
Borel measurable and has finite range. Consequently, the game has a value, and each player has a strategy that is optimal 
in every subgame. 


Proof. Take such an ordinal € and a history h € H. The game G;(h) is a game in the sense of Section 2, once 
we adopt the following convention: once Player I accepts a proposal by Player II (let us denote this action by 
a‘), the game continues, Player I being the active player henceforth, and the only available action being a*. With 
this convention, a play in G;(i) has either the form 


((ho, a0), Por (1,41), Pis+++) (5) 
or the form 
((horao), Por- +++ (Apra), Per dO ,...), (6) 


where the actions (H,,,4,,) and p,, of G:(h) obey the rules of the game as described above. 


m/s 
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Let Z denote the set of plays in G;(h). For each z € Z let f(z) denote the outcome of the game G;(h). Thus 
f(z) is the unique play that extends the histories h = họ < h <--- if z is of the form (5), or f(z) =p, if z is of 
the form (6) for some k € N. Then Player I's payoff function v in the game G;(h) is given by v(z) = um (f (2)) for 
each z € Z, that is, v is a composition of f and u,n 

We endow Z with the topology as in Section 2. Then f: Z — P is a continuous function: indeed, it is easy 
to see that if a sequence (Z„)„en converges to z then the sequence (f(Z,,)) nen converges to f(z). It follows that 
f is Borel measurable. Now recall that the function u,n) is assumed to be Borel measurable. It follows that v 
is Borel measurable, being a composition of Borel measurable functions. Since u, is further assumed to have 
finite range, the function v has finite range. Now the result follows by Lemma 2.1. O 


Notice that if the statement ®(€) is true, then the game G;(h), the quantities a;(h), B<,,(1), and the sets 
P<,,(h) are all well defined for each h € H. 


Lemma 3.2 (Monotonicity of the Sequences). Let € be an ordinal such that the statement ®(€) is true. Then, for each 
history h € H it holds that 

(a) P,(h) 2 P<(h) for every ordinal A < &, 

(b) a,(h) < c(h) for every ordinal A < €, 

(c) Be(h) < aclh) = Bes (1). 


Proof. We first prove claims (a) and (b). Both claims are obvious if A =0. Thus, suppose that A >0. As A < é, 
we have 
palh) = max a, (It) < max @,(h) = B<(h) 
nN</ PE: 5 j 


for each h € H. Therefore, (a) follows. Now consider an optimal strategy for Player I in the game G,(h). By (a), 
this same strategy can also be used in the game G;(h), while at the same time any strategy for Player II in the 
game G;(h) is also a legal strategy in the game G,(h). It follows that Player I can guarantee a payoff of at least 
a,(h) in the game G;(h). Therefore, (b) holds. The claim (c) follows directly from (b). o 


We argue that there exists an ordinal &* such that 
P;.(h)=P,(h) for allheH and all A>. (7) 


Now, if P:(h) =Ø for some h € H and some ordinal €, then by our convention P,(i) = @ for all histories h and 
all A > č. Suppose that P;<(h) # @ for all h and all €. Then in view of Lemma 3.2 there exists* an ordinal €* such 
that P,.(h) = P:.,,(h) for every h € H. Finally the definitions imply that &* satisfies (7). Henceforth we let ¢* be 
the least ordinal satisfying (7). 


3.3. Examples 

We illustrate the algorithm by means of two examples. The first example comes from Flesch et al. [9]. The game 
has no SPE,’ and the algorithm of Section 3.2 outputs a collection of empty sets. The second example, which 
admits a unique SPE, demonstrates that the algorithm might take more than w steps to terminate. 


Example 3.3 (Flesch et al. [9]). Consider the following two-player game. The action set is A = {1,2}. Player 1 
starts the game. The active player decides who the next active player is by choosing the corresponding action. 
Let Sọ denote the set of plays where both actions 1 and 2 are played infinitely many times, S, the set of plays 
of the form h1” =(h,1,1,1,...) for h € H, and S, the set of plays of the form h2” =(h,2,2,2,...) for h € H. The 
payoffs are (0,0) for any play in Sọ, (—1,2) for any play in S,, and (—2,1) for any play in S. 

The values P.(h) and a,(h) are given by the Table 1, where S,() = S, N P(h). The algorithm terminates after 
four rounds with an empty set of plays at each history. Corollary 4.2 thus implies that the game has no SPE. 

The entries of the table are obtained as follows. In view of item (3) of Lemma 3.2, the calculation of 6;(h) is 
straightforward, so we focus on the computation of P;(h) and a;(h). 

Step 0. By definition. 

Step 1. We have P,(h) = P(h) by definition. Consider player 1’s history h. In the game G,(i) Player I, who 
represents player 1 of the original game, “realizes” that he can guarantee the payoff of —1 by always taking 
action 1. Player II makes sure that Player I’s payoff is not greater than —1 by proposing, after each move (h, ax) 
of Player I, the play (h;,4,)2”. This implies a,(h) = —1. 

Consider player 2’s history h. In the game G,(h) Player I (who is essentially representing player 2 of the 
original game) can guarantee the payoff of 1 since he can induce the play 2°. Player II makes sure that I’s 
payoff is not greater than 1 by proposing the play (h;,a,)2~ after each move of Player I. We obtain a,(h) =1. 
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Table 1. The iterations in Example 3.3. 


Beh) P,(h) a(h) 
č ı(h)=1 ı(h)=2 ı(h)=1 uh) =2 uh) =1 uh) =2 
0 —2 0 P(h) P(h) —2 0 
1 —2 0 P(h) P(h) -1 1 
2 -1 1 S (h) {42°} U S; (h) —1 2 
3 —1 2 S (h) S,(h) 0 2 
4 0 2 @ @ * * 


Step 2. For each player 1’s history h we obtain P,(h) = S;(h). Consider player 2’s history h. The set P,(h) 
consists of the play h2” and all plays of the set S,(/). Notice that h2% is an element of P,(h) since only player 2 
is active along this play after history h. 

Consider player 1’s history h. We have a,(h) = -—1 as in Step 1. 

Consider player 2’s history h and the corresponding auxiliary game G,(h). Player I realizes that, if he plays 
action 1, he is proposed a play in the set S,(h,1). Accepting such a proposal yields him the payoff of 2. This 
leads to a,(h) =2. 

Step 3. For a player 1’s history h we obtain P3(h) = S,(h). Consider player 2’s history h. The play h2% is not 
in P,(h) since it violates the beta value for player 2, which has been updated to 2. Consequently we have 
P3(h) = S,(n). 

Consider player 1’s history h and the corresponding auxiliary game G;3(/). Suppose Player I’s makes the first 
move (hp, 49) = (h,2). He is then offered a play py, an element of the set P3(h,2) =5,(h,2). The play py could be 
written in the form h,1° for some player 1’s history h, extending (họ, 4o). Consequently Player I could deviate 
from pọ by playing (h,,a,) =(h,,2). He is then offered a play p; € S,(h,,2). He could likewise deviate from py. 
Continuing this way, Player I induces a play in which both actions 1 and 2 appear infinitely often, yielding him 
the payoff of 0. Thus a,(h) =0. 

For player 2’s history h we have a3(h) =2, since a(h) =2. 

Step 4. It is clear that there are no plays satisfying the beta levels of 0 and 2 for players 1 and 2, res- 
pectively. < 


Example 3.4. Consider an infinite centipede game with infinitely many players in which player 0 plays once, 
and all other players move infinitely many times. All payoff functions are binary and are upper semicontinuous. 

The set of players is I =N. Let N* =N\{0}. Take any infinite partition {S': i € N*} of N* such that each S' is 
infinite. Player 0 is active in period 0 and each player i € N* is active at the infinitely many periods belonging 
to S'. The action set is A = {s,c}, where s stands for stopping the game, and c for continuing. Player 0’s payoff 
is 0 if any player stops, and 1 if no one stops. Player i € N* gets a payoff of 0 if a player j <i stops the game, 
and his payoff is 1 otherwise. 

The algorithm is illustrated by the table below. The rows correspond to the iteration steps. The first column 
shows the complement of P;(@), and the numbers in the remaining columns show the security levels a;(h) for 
the respective players, for nonterminal histories h, that is, histories where no one yet has stopped the game. We 
let B; be the set of plays where player i is the first player to stop. 

The calculation of the values in Table 2 is straightforward. Consider iteration step € = 1. Take a player 1’s 
history h. Then Player I (who represents player 1 of the original game) can guarantee the payoff of 1 in the 
game G,(h) by never stopping. Thus a,(/) =1. Take a player i’s history h where i > 1. Now Player I (representing 


Table 2. The iterations in Example 3.4. 


P\P.(@) uh) =0 uh)=1 uh) =2 uh) =3 
€=0 Ø 0 0 0 0 
é=1 Ø 0 1 0 0 
Ec B, 0 1 1 0 
€=3 B, UB, 0 1 1 1 
=w B,UB,U--- 1 


E=wtl B, UB, UB,U-* 1 1 1 1 
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player i of the original game) cannot guarantee 1, since after any period where player i is active, eventually 
there comes a period where player 1 is active, and if player 1 stops, player i gets payoff 0. Thus a,(h) =0. 
Likewise, a,(@) =0. 

At iteration step € =2, player 1 never stops. Starting with player 2, the analysis is similar to that of the 
previous paragraph. That is, for any history h of player 2 we have a,(h) =1, and for any history h of player i > 2 
we have a,() =0. Likewise, a,(@) = 0. 

A similar argument holds for each € < w. 

At iteration step € = w, we obtain B,,(@) =0 and 6,(h)=1 for any history h other than Ø. As a consequence, 
no player i € N* stops. Player 0, on the other hand, is still allowed to stop since f,,(@) =0. 

At €=w +1, player 0 can guarantee 1 by playing continue. The iteration reaches a fixed point at ¢* = w +1. 
Note that P;.(@) is the singleton consisting of the play (c,c,c,...). And indeed, in the unique SPE, each player 
plays continue at each of his histories. < 


It is not difficult to show, by means of examples like the one above, that the ordinal €, can be arbitrarily large. 
Examples of this sort have been previously constructed in Flesch et al. [7] and Flesch and Predtetchinski [11]. 


4. Overview of the Main Results 
In this section we provide an overview of the main findings of the paper. We first state two main results, and 
then turn to the discussion of various additional results. 

Our first result is a characterization of SPE plays in terms of the limit of the recursive sequence as defined in 
the previous section. The proof is delegated to Section 5. 


Theorem 4.1 (A Characterization of SPE Plays). Consider a perfect information game Q. Assume that the payoff function 
of each player has finite range and is Borel measurable. Then P*(h) = P;.(h) for every h € H. In particular, for a play p €P, 
there exists an SPE o such that p = n(o, ©) if and only if p € P;.(@). 


Corollary 4.2. Consider a perfect information game Q. Assume that the payoff function of each player has finite range 
and is Borel measurable. Then the game has an SPE if and only if P;.(@) + Ø, or equivalently, if and only if P:(h)#@ 
for each history h € H and each ordinal €. 


We now turn to our second contribution, the existence theorems for games with upper semicontinuous 
payoffs, which is from a technical point of view more demanding than the first. We apply Corollary 4.2 to 
address an open problem raised in Purves and Suddeth [24]. The authors show that there is an SPE provided 
that each player’s payoff function has finite range and is upper semicontinuous, and that the number of players is 
finite. They further raise the question of whether one could dispense with the second assumption, the finiteness 
of the player set. We answer the question affirmatively. 

The technique employed by Purves and Sudderth is based on the induction on the number of distinct payoff 
vectors in the game. This feature of the proof makes it difficult to extend it beyond the case of finitely many 
players. Instead, we show that in games where each player’s payoff function has a finite range and is upper 
semicontinuous, our algorithm returns a nonempty set of plays at each step of the iteration. This is Theorem 4.3. 
Corollary 4.4 then follows by discretizing the payoff functions, exactly as is done in Purves and Sudderth [24]. 


Theorem 4.3 (Existence of SPE in Upper Semicontinuous Games). Consider a perfect information game. Assume that 
the payoff function of each player has finite range and is upper semicontinuous. Then, P:(h) # Ø for each history h € H 
and each ordinal €. Consequently, the game admits an SPE. 


Corollary 4.4. Consider a perfect information game. Assume that the payoff function of each player is bounded and upper 
semicontinuous. Then, for every e >0, the game admits an e-SPE. 


The assumption of upper semicontinuity implies that the sets P;(/) are all closed. This observation plays a 
crucial role in the proof of Theorem 4.3. The proof is delegated to Section 6. 

We presently turn to additional results. Our goal is to evaluate to what extent our results depend on the 
assumption that every player’s payoff function only takes finitely many values. Therefore in the final section 
of the paper we consider a richer class of games, games where the payoff functions are only required to be 
bounded and Borel measurable. 

In this larger class of games one can still show that P;.(@) contains all SPE plays. The converse is no longer 
true. However, we are able to show that each play in P;.(@) could be induced as an e-SPE for every e > 0 
(Theorem 8.1). This discrepancy arises because in the auxiliary zero-sum game Player II might not have an 
optimal strategy. We refer the reader to Section 8.1 for a more thorough discussion of this point. 
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Our final result concerns the technique of “discretizing” payoff functions, that is, approximating a bounded 
payoff function by a payoff function having finite range. As mentioned in the introduction, the technique has 
been used frequently to reduce the problem of existence of e-SPE in games with bounded payoff functions to 
the problem of existence of SPE in a game where the payoff functions have finite range. Also in this paper, it 
is by means of a discretization of payoff functions that we pass from Theorem 4.3 to Corollary 4.4. Our result 
could be interpreted as saying that passing to discretizations is without loss of generality, at least if one is only 
interested in the question of existence of €-SPE for every e€ > 0. We refer to Theorem 8.3 for a rigorous statement 
of the result. 


5. The Proof of Theorem 4.1 
The proof of the inclusion P*(h) € P+ (h) is straightforward: we show by induction on € that if ø is an SPE, then 
n(o, h) € P;(h) for every history h € H. 


Proof that P*() C P;.(h) for each h € H. Let o be an SPE of Q. Let A(é) be the following statement: n(o, h) € 
P.(h) for every history h € H. We prove that A(€) is true for each € by transfinite induction on €. 

For € =0, the statement A(é) is true by the definition of P)(h). Now, for an ordinal € > 0, suppose that A(A) 
is true for all A < €. We need to show that A(é) is also true. 

Consider a history h € H. Take any history g € H such that h < g < n(o, h). We need to show that u,.)(7(0,h)) 
> B:(g). Notice that n(o, h) = n(o, g). Thus, in view of (2) it is sufficient to show that 


Uy (nlo, g)) 2 alg) (8) 


for every for every ordinal A < é. 

Take an ordinal A < € and consider the game G, (g). The strategy profile o induces a strategy Ay for Player II 
in the game G,(g), as follows: after Player I’s move (h,,a,) € H, let Player II propose the play p; = n(o, (hy, a,)). 
This is a legal strategy for Payer II since n(o, (h;.,a,)) € P, (hy, ap) by the induction hypothesis. Let A; be a strategy 
of Player I in the game G,(g) that makes the initial move (ho, 4), where hg = g and ay = 0,(g), and then accepts 
the first proposal of Player II. Clearly, using A; against the strategy A, results in the play n(o, g) being proposed 
and accepted. Since under the strategy Ay Player II essentially proposes to follow the SPE strategy profile o, 
Player I can gain no more than u,.(7(0,g)) by playing against Ay. Hence, A; is a best response to Ay. The 
inequality (8) follows. 

This proves that the statement A(€) is true. oO 


The intuition behind the proof of the inclusions P*(h) 2 P;.(h) is as follows. We first prove the inclusion for 
h = Ø and then generalize to an arbitrary history h. 

If p € P:.(@), we can define an SPE o thus: from the root of the game Q, follow the play p, as long as no 
player deviates. If a player i deviates from p at a history họ by playing action a), choose some strategy oe) 
for Player II in the auxiliary game G¿.(họ) that is optimal in every subgame (recall that such a strategy exists 
by Lemma 2.1). Then o prescribes to follow the play py proposed by Player II under the strategy pa after the 
history (o,ao). Suppose that some player deviates from pọ at a history h, with an action a4. If it is player i 
again, then o prescribes to follow the play p, proposed by Player II still under the strategy guy after the history 
((A9, 4), Po, (My, 41)). If the deviating player is other than i, choose some strategy Eh for Player II in the auxiliary 
game G;.(h,) that is optimal in every subgame. Then o prescribes to follow the play p, proposed by Player II 
under his strategy = after the history (h,,a,). The process thus continues ad infinitum. 

Before we proceed to the proof, we recall that the set P;.(i) is nonempty for some history h if and only if 
P;.(h) is nonempty for every h. We can thus assume that P; (h) + Ø for every h € H, for otherwise there is 
nothing to prove. 


Proof that P*(2) 2 P;.(@). Take a p € P;.(@). We construct an SPE ø in Q such that p = n(o, Ø). 

For each history g € H, fix a strategy Lf, for Player II in the game G; (g) as in Lemma 3.1, i.e., a strategy that 
is optimal in every subgame of G;.(g). 

Define a strategy profile o in Q as follows. The players are to follow the play p. Suppose that the players fol- 
low p until history ho, with hy < p, where the first deviation occurs, by player (họ) taking action ag incompatible 
with p. Let pọ = EH (ho, a0). Starting with the history (ho, 4o), the players are to follow the play po. 

Consider a sequence ((ho, 49), Po,- --, (hk, ak), Pr), Where (Ng, do),-.-, (My, 4,) are successive deviations and for 
each m, the play p,„ is the play that o prescribes the players to follow starting with the history (H,,,,4,,). Suppose 
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that the next deviation occurs at a history hg, with (hy,4,) < hgy < Pr, and let a,,; be player 1(h,,,)’s action 
at h,,, not compatible with p,. Let £ be the least element of {0,...,k +1} such that i(h,) =--- = 1(h,,,). Define 
Pra = By (hera), Per- -, gy) Prr Orava): Then, the players are to follow the play p,,, starting with the 
history (h;,1,4,4,). This completes the definition of the strategy profile ø. 

By construction, n(o, Ø) =p. We argue that o is an SPE in Q. 

First notice that for every history g € H we have n(o, g) € Pa (g). To see this, first suppose that g is a prefix 
of p. Then n(o, g) =p. Since p is an element of P;.(@) and g is its prefix, p is also an element of P:.(g), as desired. 
Suppose now that g is not a prefix of p. Then to g there corresponds a sequence ((ho, 4o), Po,- - - , (Mz, 4x), Pr) as 
above such that (h,,a,) is a prefix of g and n(o, g) = p,. Since p, is an element of P;.(h,,a,) and g is its prefix, 
p, is also an element of P;.(g), as desired. 

Now, take any g € H. As P;.(g) = P.,;(g), we have urg (nlo, 8)) = Be-ui(@) =aelg). Thus player (g) obtains at 
least a;.(¢) by playing in accordance with o in the subgame of Q starting with g. On the other hand, following 
a deviation by player (g) at g, the strategy o essentially replicates the behavior of Player II in the game G;.(g), 
so that the deviation yields no more than a;.(g). We conclude that ø is an SPE. o 


Proof that P*(i) 2 P;.(h) for each h € H. Now take a history h € H of length t and a play p € P;.(h). We construct 
an SPE ø such that 7(0,h) =p. 

An argument similar to that above could be used to show that the subgame of Q starting at history h has 
an SPE o” that induces the play p. Likewise, for each history h’ € H of length t, the subgame of Q starting 
at history h’ has an SPE o". Finally, we can use backward induction starting at period t to build a strategy 
profile ø that is an SPE of Q and that agrees with the strategy profiles o” for every h’ of length t in the subgame 
following h’. In particular, n(o,h)=p. o 


6. The Proof of Theorem 4.3 
Throughout this section, we assume that for each i € I the function u; has finite range and is upper semicontin- 
uous. The crucial consequence of this assumption is that the sets P;(i) are closed. 


Lemma 6.1 (Closedness). Let € be an ordinal such that the statement ®(€) is true. Then, for each history h € H, the 
set P;(h) is closed. 


Proof. The set P:(h) can be expressed as an intersection of the set P)(1) and sets of the form {p € P: u,(p) > x} 
where i is a player and x a real number. Since the functions u; are upper semicontinuous, all of such sets are 
closed. As the intersection of closed sets is closed, the result follows. o 


For an ordinal € let () denote the statement that for each h € H there exists an action o;(h) such that 
(h,o<(h)) €H and 
P,(h) 2 P<(h,o;(h)). (9) 


We shall prove that the statements ®(€) and (é) are true for every ordinal € simultaneously by transfinite 
induction. As remarked earlier, ®(0) is true. Clearly (0) is also true. As c(h) one could take an arbitrary 
action a such that (h,a) € H. 


Lemma 6.2. Let & be an ordinal such that the statements P(&) and Y(E) are both true. Then, for each h € H, the play 
n(oz,h) is an element of P;(h). 


Proof. Take a history h € H. Let p=7(o;,h) and let hy,h,,... be the enumeration of the successive prefixes of p 
starting with ho =h. By (€), the set P;(h,) is nonempty for each t €N. By (9), we know that Pe(ho) 2 Ps(h,)2---. 
Now take any p, € P;(h,) for every t € N. Then p, € P;(h). Clearly, the sequence {p,} converges to the play p. 
Since P¿(h) is a closed set by Lemma 6.1, we conclude that p € P;(h), as desired. o 


Lemma 6.3. Let & be a limit ordinal such that the statements P(A) and (A) are both true for each A < &. Then, ®(&) 
and Y(&) are also true. 


Proof. Take a history h € H. Define recursively the sequence hy,h,,... of histories and the sequence of ordinals 
Ao, A4,... as follows. Let hy = h. Suppose that the history h, has been defined. Let A, < € be a successor ordinal 
such that 

By, (Me) = Bz (hy). 
If k > 0, we require in addition that A,_, < A. To see that such an ordinal exists, notice that by (2), the sequence 
(B,(hy))ace is nondecreasing, and max, << 6, (M;,) = B: (hy). Hence, there is an ordinal A, < € such that 6 ,(h,) = 
B<(h,) for each A with A, < A < €. This implies that there does exist an ordinal A with the desired properties. 
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Let hks = (Ap, 0,,(h;)). This completes the recursive definition. 
Now for each k € N, we have the inclusions 


Py (he) 2 Pa. (esr) 2 Papa Mest), (10) 


where the first inclusion is by (9), and the second inclusion holds by A, < A,,, and by Lemma 3.2. Hence, 
for k,l EN 
Py (hy) 2 Pa,(he), whenever k < £. (11) 


Let p be the unique play extending the histories hy < h, <---. We argue that p € P;(h). 

For each £ €N, let p; be an arbitrary element of the set P, (h,). Since h; is prefix of pẹ, the sequence {pe} en 
of plays converges to the play p. Now fix a k € N. We know by (11) that pẹ € P,,(4,) for all € > k. As Pj, (h,) is 
a closed set by Lemma 6.1, we conclude that p € P),(h,). Thus uma (P) = Pa, (hy) = Be(hy). Since this is true for 
each k €N, we conclude that p € P;(h). 

This proves the statement ®(€). 

To prove the statement ‘Y(€), let o; (ho) = 0), (Mo). We show that P; (ho) 2 P:(Ng, 0: (M9). Take a q € Pz (ho, 6:(Mp)). 
To prove q € P:(hp), it suffices to show that u,;,)(q) = Pe(ho). We have P, (ho) 2 Pa, (h1) 2 P:(hi) = Pe(g, 6: (No)), 
where the first inclusion comes from (10), and the second one is by monotonicity. So q € P} (họ) and hence 
U(n)(9) = Ba, (ho) = B: (ho), as desired. 

This proves the statement Y(&). o 


Àk 


A 


We now proceed with the more challenging part of the proof: showing that if (€) and Y(&) are both true 
for some ordinal €, then ®(€ +1) and V(é +1) are also true. 

Let € be an ordinal such that the statements ®() and Y(¢&) are true. Take an h € H. Fix some o;(/) satisfy- 
ing (9). Also fix some optimal strategy £! for Player I in the game G;(h). Suppose that (h,a) is the first move 
Player I makes under the strategy L!. Define oz, (h) =a. 

Given a set F CI of players, let of (h) denote o;(h) if (h) ¢F and denote o¢,,(h) if (h) € F. Thus, in particular, 
o2(h) = c(h) and ot (h) = 0:,(h). Define also 


for every g € H with h < g <p 
Pi(h)=4peP(h): if (g) €F then uy. (p) > ac(g) 
if (g)¢F then Uy (p) > felg) 


Thus, in particular, P?(h) = P:(h). In view of Claim (3) of Lemma 3.2, P:(h) = P;,,(h). By the same claim, if 
F CECI then P(h) 2 Pz(h). As in Lemma 6.1 the sets P£(h) can be shown to be closed. 


Lemma 6.4. Let € be an ordinal such that the statements ®(€) and Y(&) are true. Then, for each h € H and each F Cl, 
it holds that PE(h) =) Ph, o£ (h)). 


Proof. Take a p € P£(h,o{(h)). We distinguish two cases, depending on whether 1(/) is an element of F or not. 

Suppose first that 1(h) ¢ F. In order to show that p € P/(h), it suffices to prove that u, (p) > B:(h). We have 
o1(h) = o(h). Moreover, P:(h) 2 P<(h, o;(h)) 2 PE(h, o¿(h)), where the first inclusion is (9). Hence p € P:(h) and 
thus u)(p) 2 Be(h). 

Suppose now that (h) € F. In order to show that p € P: (h), it suffices to prove that um (p) = œ:(h). We 
have of (h) = o<,,(h). Now recall that X? is an optimal strategy for Player I in G(h) such that its first move is 
(h, 0¢,;(h)). 

Suppose that, following Player I's move (hg,4d9) = (h, o;,,()), Player II proposes the play pọ = p. This is a 
legitimate move by Player II since py € PE (hg, a0) S P-(Ng, a0). Consider Player I’s reaction to po. 

If, under the strategy £}, Player I accepts pọ, we must have u,q)(po) = as(h), for otherwise X! would not be 
optimal. Now suppose that, under the strategy £!, Player I responds to pọ by making a move (h4, a1) € H. From 
Player I's point of view, what follows then is strategically equivalent to what happens in the game G;(h,) after 
Player I makes the move (h,,a,). Hence, following (h,a), Player II can make sure that I’s payoff is not greater 
than a;(h,). Since Z} is optimal, we must then have a;(h,) > a;(h). Since (ho, ao) < h, and since py € Pi (hg, ao), 
we also have u,)(Po) = @:(h,). It follows that u,,)(po) = a<(h), as desired. o 


Lemma 6.5. Let € be an ordinal such that the statements P(&) and Y(&) are true. Let F CI be a finite set. Then, for each 
h € H, the play n(oz,h) is an element of PẸ(h). 
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Proof. The proof is inductive on the cardinality of F. Lemma 6.2 implies that the result holds for the set F = Ø. 
Suppose that the statement has been proven for each set F CI of cardinality n. Consider a set E CI of cardinality 
n +1. We prove that for each h € H the play n(o¥,h) is an element of PE(h). 

Take a history h € H. Let p = n(oẸ,h) and let hy,h,,... be the enumeration of the successive prefixes of p 
starting with họ =h. In view of Lemma 6.4, we have the indusions PË(ho) 2 Pi(h,) 2---. Therefore, it suffices to 
show that there is a t €N such that p € PF (h,). 

Take an arbitrary player i € E. We consider two cases, depending on whether player i controls finitely or 
infinitely many prefixes of the play p. 

Suppose first that player i controls only finitely many preies of the play p. Let K € N be such that none of the 
prefixes hy,hx,,,... are controlled by player i. Then p> n(o, hy)= Tor, Nx ), where F = E\{i}. By the induction 
hypothesis the play p = n(of z/Mx) is an element of PE (hx). Again using the fact that none of the prefixes of p 
after time K are controlled by player i, we conclude that p € P& (hx). 

Suppose now that player i controls infinitely many prefixes ‘of the play p. Let h,,,h,,,... be the subsequence 
of the sequence /iy,h,,... of the prefixes of p that are controlled by player i. We let g, =h, for each k € N and 
we let a, denote the action of p following the prefix gp, that is, (8x, 4%) = (hiap) = My, 41. Define 


a*=limsupa;(g,) and f° =limsupf;(g;). 
k-00 k=% 

Since the sequences a;(g,) and f(g) only assume finitely many values, there exists a K € N such that a* > a;(g;) 

and f° > B;<(g;) for each k > K. 

We show that p € P£(gx). 

It holds that u;(p) = a:(g,) for every k = K. To see this, notice that for each k there is a play q; > g, such that 
u;(q,.) = &:(g;). Such a play can be obtained as an outcome in the game G;(g,) where Player I follows an optimal 
strategy. Notice that the sequence {q,} converges to p as k tends to infinity, and that upper semicontinuity of 
the function u; implies that u;(p) > a*. The claim follows. 

Thus, it remains to be shown that p € PESK) with F = E\{i}. 

Notice that for each k € N, we have 

PE (Ser 44) 2 Ps (Sis): (12) 

To see this, recall that (9,,4,) = h; +1 and that g} =h;,,,. Among the histories h,,,,,...,/;,,,-1, none belongs to 

player i. Hence, for each t € {t +1,...,t,,, — 1}, it holds that hig = (m, 05 (hy )= (hy, o£ (h, )). Thus, Lemma 6.4 

implies that P: (h,) 2 P; (h1). By ipes dios the inclusions we obtain PË zga) 2 PE z (Ar); which is exactly (12). 
Define 


teal! 


B ={q €P: u,(q)> B°}. 


Notice that the set B is closed since u; is upper semicontinuous. We next argue that 
PEC) OB 2 PE(gu a) NB, for each k > K. (13) 


To see this, take a k > K and a q € PECS Ax) NB. As u;(q) = B* = B<(g,), and since 1(g;,) =i is not an element of F, 
we have q € PË(g,), as desired. 
Combining (12) and (13), we obtain the inclusions 


PES) OB 2 PEC) NB, for each k > K. (14) 


Finally, observe that B<(g,) = 6° for infinitely many k, and that for each such k it holds that Pf(g,) S B. 
As Pf(g,) is nonempty by the induction hypothesis, we conclude that P z(8;) N B is nonempty for infinitely 
many k > K, and hence in view of (14) for each k > K. For each k > K, take an arbitrary element q, € PE (&) NB. 
Then q; € Pi (gx) MB by (14). Since both Pf(g,) and B are closed sets and the sequence {q;} converges to p, we 
conclude that p € P:(gx) OB. This completes the proof of the induction step. O 


Lemma 6.6. Let € be an ordinal such that the statements ®(€) and Y(&) are true. Then, for each h € H, the play Tr, h) 
is an element of P; (h). 


Proof. Let p = n(o, h) and let hy,h,,... be the enumeration of the successive prefixes of p starting with hy =h. 
For k EN, let I, = {t(ho),..., e(h,)} and let p; = no ,h). By Lemma 6.5, p; € PS ‘(h). Notice moreover that hy < py 
for — k €N. Hence, the Pr {pr} converges to the play p. Now fix an m €N. For each k > m it holds that 
I, 21,,, and hence P(h) c P” (h), so that p, € P (h). As P (h) is a closed set, we conclude that p € P i" (h). In 
particular, un, (P) = &s(hn). Since this is true for each m €N, it follows that p € P! z(h). m| 
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Lemma 6.7. Let & be an ordinal such that the statements B(€) and V(é) are true. Then, B(E +1) and Y(& +1) are 
also true. 


Proof. Recall that bets) = P;,,(h) for each h € H. Now the statement ®(€ + 1) follows by Lemma 6.6 and the 
statement Y(€ +1) by Lemma 6.4. o 


Combining Lemmata 6.3 and 6.7, and the fact that (0) and (0) are true, we obtain the following result. 
Lemma 6.8. For each ordinal €, the statements D(€) and Y(&) are true. 


Thus, in particular P;.(@) + . Theorem 4.3 now follows from Theorem 4.1. 


7. Special Cases 
In this section we discuss a number of interesting special cases of our model. We assume throughout that each 
player’s payoff function has finite range. 

Games satisfying the one-deviation principle. We say that a strategy profile is one-deviation immune (ODI) 
if no player can improve his payoff by deviating at a single history. Formally, the strategy profile o is ODI if for 
each (h,a) €H it holds that u”,)(0,h) = um (0, (h,a)). Each SPE is clearly an ODI strategy profile, but in general 
the converse is false. A game is said to satisfy the one-deviation principle if the set of its SPE equals the set of 
ODI strategy profiles. 

The importance of ODI strategy profiles for our work stems from the results of Flesch et al. [7], where the 
authors define an algorithm terminating with the set of plays that could be supported by an ODI strategy profile. 
This result thus provides a characterization of SPE plays in any game satisfying the one-deviation principle. 
Furthermore Flesch et al. prove® existence of an ODI strategy profile under the assumption that the number of 
players is finite, and show, by means of an example in Section 4.3, that this assumption cannot be dropped. 

The one-deviation principle is satisfied if each player i’s payoff function is lower semicontinuous in player i's 
own actions. This means that liminf,_,,, u;(p,,) = u;(p) whenever {p,,} is a sequence converging to p such that for 
each n €N either p, =p,4; or the longest common prefix of p, and p,,,, is controlled by player i. In particular, 
player i’s payoff function is lower semicontinuous in player i’s own actions if (1) it is lower semicontinuous 
in the sense of Section 2, or (2) if for each play p the set of prefixes of p where i is active is finite. Thus the 
one-deviation principle is satisfied in a game where the player set is I = N, and player t is active at each history 
of length t. 

We presently turn to two important subclasses of games satisfying the one-shot deviation principle: the class 
of games with continuous payoff functions (falling under condition (1) above), and the class of centipede games 
where each player acts exactly once (falling under (2) above). 

Games with continuous payoffs. Of particular importance for economic applications are games where the 
payoff functions are continuous. Any such game satisfies the one-deviation property. Notice that a function 
having a finite range is continuous if and only if each play p has a prefix h such that the function is constant 
of P(h). Consequently, when the number of players is finite, one could prove existence of SPE by backward 
induction starting with subgames where every player’s payoff function is constant.’ This technique runs into 
difficulties when the number of players is not finite, since then there might be no such subgame. However, 
whether the player set is finite or not, a slight modification of the algorithm in Flesch et al. [7] lends itself to a 
proof of the existence of SPE (see Flesch and Predtetchinski [11)). 

We remark that payoff continuity, as defined here, is weaker? than the condition of continuity at infinity 
as defined in Fudenberg and Levine [13] and in Carmona [5]. The two conditions are equivalent if the set of 
actions is finite. 

Infinite centipede games. Cingiz et al. [6] consider infinite centipede games where each player acts exactly 
once. Assuming that the payoff functions are upper semicontinuous, the authors show that the game admits an 
SPE. In this setup the method of finite truncations of the game tree suffices. 

Games with upper semicontinuous payoffs and finitely many actions. Finally, we consider the class of 
games where each player’s payoff function is upper semicontinuous and the set of actions is finite. In this class, 
one could give an alternative proof of the existence of SPE, taking the concept of ODI as a starting point. While 
this result is subsumed by Theorem 4.3, we believe that the proof warrants some attention. The main idea is to 
consider ODI strategy profiles with respect to a certain refinement of the original payoff functions. 

For simplicity, we assume that each player’s payoff function takes integer values (recall our assumption that 
the range of the payoff function is finite). For each m € Z, let V,(m) = {p € P: u,(p) => m}. We use the metric d on 
the set of plays as defined in Section 2. For a play p € P and a nonempty subset of plays Q € P we let d(p,Q) 
denote the distance from p to a set Q, i.e., the infimum of d(p,q) over q € Q and we let d(p,@) =0. 
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For a player i € I define 
vi(p) = ui(p) — 34(p, Vi(u,(p) + 1). (15) 


The payoff function v; could be thought of as a “refinement” of u; that breaks indifferences by assigning a 
higher rank to plays that are closer to the set of plays with the higher utility. Indeed, notice that v;(p) < 0,(q) 
precisely when either (A) u,(p) < u,(q) or (b) u;(p) =u,(q) =m and d(p,V,(m +1)) > d(q,V,(m +1)). 

Furthermore, v; attains a minimum over any nonempty subset of plays, since u; has a finite range of integer 
numbers, while d takes values in the set {0} U {2-": n € N}. Finally, v; is upper semicontinuous. This follows 
from the upper semicontinuity of u; and from the continuity of d(p,Q) in p. 

It is easy to prove that, in the game © with the payoff functions (v;);e the algorithm in Flesch et al. [7] is 
well defined, and returns a nonempty subset of plays at every step. It is at this step of the proof that we need 
to assume the set of actions to be finite. Consequently, the game admits an ODI strategy profile. Moreover, each 
ODI strategy profile in Q’ is an SPE in the original game Q, with the additional term involving the distance in 
Equation (15) making sure that deviations at infinitely many histories are not profitable. 


8. On the Assumption of Finite Range 

Until now we have performed our analysis under the assumption that every player’s payoff function only takes 
finitely many values. In this section, we turn to a richer class of games, the games where the payoff function 
is only required to be bounded and Borel measurable. In Section 8.1 we reconsider the characterization of SPE 
plays, and in Section 8.2 we study discretizations of the payoff functions. 


8.1. Extending the Characterization of SPE Plays 
To accommodate for the payoff functions taking infinitely many values, we need a slight modification to our 
definitions in Section 3: replace the formula (2) by 


B<(h) = sup a,(h). 


A<& 


Theorem 8.1. Consider a perfect information game Q. Assume that the payoff function of each player is bounded and 
Borel measurable. Then P*(@) € P;.(@). Conversely, for each p € P;.(@) and each e >0 there exists an e-SPE o such that 
p = n(o, Ø). 


The proof of the first claim of the theorem is similar to the first part of the proof of Theorem 4.1 in Section 5. 
The proof of the second claim is similar to the second part of the proof of Theorem 4.1, with the difference that 
one lets Lf be any Player II’s strategy that is e-optimal in every subgame of G+ (g), where € > 0. Such a strategy 
exists in view of Lemma 2.1. 

Theorem 8.1 asserts that P;.(@) contains all SPE plays and is contained in the set of e-SPE plays for each 
e >0. The discrepancy between P;.(@) and P*(@) stems from the fact that in the game G;.(g) Player II might 
have no optimal strategy. This point is further illustrated by the example below. The game considered in the 
example has no SPE, so that P*(@) = Ø, whereas P;.(h) is not empty for each history h. 


Example 8.2. This is a modification of Example 3.3. There are two players, and the action set is A = {1,2}. 
Player 1 starts the game. The active player decides who the next active player is by choosing the corresponding 
action. Let Sọ denote the set of plays where both actions 1 and 2 are played infinitely many times, 5, the set of 
plays of the form h1% =(h,1,1,1,...) for h € H, and S, the set of plays of the form h2% =(h,2,2,2,...) for h € H. 

The payoffs are (—1,2) for any play in S,, and (—2,1) for any play in S,. Player 2’s payoff on Sy is identically 0. 
Player 1’s payoff function on Sọ is defined as follows. For a play p = (ag, 41, - . . ) in Sy consider all blocks of consec- 
utive 2’s, and denote the lengths of these blocks by mọ, mı, ... . Thus for instance, if p =(1,2,2,2,2,1,2,2,1,...) 
then mọ = 4, m; = 2, etc. If the sequence {m,,},,cx is not eventually constant we let u,(p) =0. If it is eventually 
constant we let u,(p) = —1 +2™ where m, =m for n large. 

The essential feature of this game is that within the set Sọ player 2 can force player 1’s payoff arbitrarily 
close to —1. To do so, every time that player 2 becomes active, he should take action 2 exactly m times before 
playing action 1, where m is sufficiently large. Notice however that player 2 cannot make sure that 1’s payoff is 
exactly —1. 

The values P;(h) and a;(h) are given by the table below, where S,(h) = Sı O P(h), and h2% denotes the play 
(h,2,2,...). The algorithm terminates after three rounds, with P+ (h) = 5S,(h) for each history h. 

For Steps 0, 1, and 2 the computation is similar to that in Example 3:3: 
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Table 3. The iterations in Example 8.2. 


P.(h) a.(h) 
č ı(h)=1 uh) =2 uh) =1 uh) =2 
0 P(h) P(h) -2 0 
1 P(h) P(h) si 1 
2 S,() {n2°}US,(h) -1 2 
3 5,(h) 5,(h) -1 2 


Step 3. We obtain P3(h) exactly as in Example 3.3. Consider player 1’s history h and the corresponding 
game G;(h). Player II, for every m €N, has a strategy that ensures that Player I’s payoff is not greater than 
—1+2°": namely, proposing at stage k the play p, =(h,,a,)2""1%, i.e., the play where after (h,,a,) there follows m 
times action 2, after which only action 1 is played. Consequently a3(1) =—1. Notice that Player II has no optimal 
strategy. 

We thus conclude that ¢* =3 and P;.(h) =5$,(h) for all h € H. 

The game has no SPE. For suppose to the contrary and let o be an SPE. Applying Theorem 8.1 to the subgame 
of Q starting at history h, we deduce that n(o, h) is an element of P;.() =S,(h) for each history h. This implies 
that under the strategy oz, player 2 eventually plays action 1. Let n, be player 1’s strategy whereby action 2 
is played at each of player 1’s histories. The profile of strategies (n4, o2) induces a play in Sọ, and hence gives 
player 1 a payoff strictly greater than —1. We conclude that a deviation to 7, from g; is profitable, contradicting 
the equilibrium hypothesis. < 


8.2. On Discretizations of Payoff Functions 
This section is motivated by the technique of discretizing payoff functions: approximating a given bounded 
payoff function by a function with finite range. 

Let e > 0. Consider a payoff function u;: P —> R. A function v;: P —> R is said to be an e-discretization of u; if the 
function v; only takes finitely many values and for each p € P we have |u;(p) — v;(p)| < €. A family v = (v;)ic1 of 
payoff functions is an e--discretization of the family u =(u;);< if for each i € I the function v; is an e-discretization 
of u;. We let Q(u) and Q(v) be the games with the family of payoff functions u, respectively v. 

The result below could be interpreted as saying that passing to discretizations is without loss of generality, at 
least if one is only interested in the question of existence of subgame-perfect e-equilibrium for every positive e. 


Theorem 8.3. Consider a family of bounded Borel measurable payoff functions u = (u;)ia. The following conditions are 
equivalent: 

(a) For each e >0 the game Q(u) has an e-SPE. 

(b) For each e >0 there exists a Borel measurable e-discretization v of u such that the game Q(v) has an SPE. 


The fact that condition (a) implies condition (b) is straightforward: if v is an e-discretization of u, each SPE 
of Q(v) is 2e-SPE of Q(u). We prove the converse implication. 


Proof. Fix a subgame-perfect €/2-equilibrium o of the game Q(u). We construct a Borel measurable e-discreti- 
zation v of u such that ø is an SPE of the game Q(v). 

For a player i € I define the binary relation <; on P as follows: p <; q if there exists a history h € H with (h) =i 
and a strategy n; for player i such that o,(h) + n;(h), p = 7(0,h), and q = n(o/n;,h). Roughly speaking p <; q 
means that player i can induce a deviation from p to q. 


Claim 1. The binary relation < is transitive and well founded. 


Proof of Claim 1. We omit the subscript i for simplicity. 

To prove transitivity of <, take the plays p,q,r € P such that p <q and q <r. It is clear that p +q and q +r. 
Let the history h and the strategy n; witness the relation p <q, and let the history h’ and the strategy n; witness 
q <r. Both h and h’ are then prefixes of the play q. Hence either h’ < h or h <h’. If h’'<h<q=n(0,h’), we 
have n(o, h’) = n(o, h), implying that q =p, a contradiction. Hence h < h’. Now let n? be the strategy that agrees 
with n; on each history g such that g <h’, and agrees with 17; on all other histories. Then r = m(a/7,h) and 
ni (h) = n;(h) + 0,(h), implying that p <r. 

Suppose that there is an infinite <-descending sequence of plays, that is, a sequence {p,,},,cx, such that p,,,<Ip, 
for each n € N. For each n €N let h, be the history and nf be player i’s strategy that witness the relation 
Pnai < Pn: Using the fact that p,..< Puy and p,,, < p,, one can argue exactly as in the previous paragraph 
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that h,,,, <h,,. Since this holds for each n € N, we obtain a <-decreasing sequence of histories {h,,},,<,, which is 
clearly impossible. o 


Define the functions 


u; (p) = max{ne: n € Z,ne < u,(p)} 
u}(p) = min{ne: n € Z, u;(p) < ne}. 


It is not difficult to see that u7 and uf are both Borel measurable e-discretizations of u;. In particular, they take 
only finitely many values because the function u; is bounded. 

We define the function v;. Let S = {m(a,h): h € H,(h) = i}. On the complement of the set S we let v; be 
identically equal to u;. On the set S we define v; by recursion on the relation < (see Kunen [17], p. 103, 
Theorem 5.6), as follows: 


(16) 


u;(q) if there exists a p € S with p <; q, such that v;(p) < u; (q), 
v;(q) = A ; 
u;(q) otherwise. 


Claim 2. The function v; is a Borel measurable e-discretization of u;. 


Proof of Claim 2. The fact that v, is an e-discretization of u; follows since both u7 and uf are e-discretizations 
of u;. We show that v; is Borel measurable. 

Recall that a subset D of P is said to be discrete if each point of D is isolated, or equivalently, if each point 
p € D has a prefix h such that P(h) ND = {p}. A subset D of P is said to be sigma-discrete if it is a countable 
union of discrete sets. Each discrete set and consequently each sigma-discrete set is Borel. Moreover, each subset 
of a sigma-discrete set is sigma-discrete. It follows that any function on a sigma-discrete set is Borel measurable. 

We argue that S is a sigma-discrete set. To see this, for each k € N let S, = {n(o,h): h € H,u(h) = 
i,h has length k}. The set S, is discrete. Indeed, if h is the prefix of a play p € S, of length k, then P(h) NS, = {p}. 
It follows that S = Ujen Sx is a sigma-discrete set. 

Thus, the restriction of v; to S is automatically Borel measurable. The restriction of v; to the complement 
of S is Borel measurable since it coincides with the function u; on that set. We conclude that v; is Borel 
measurable. O 


Since o is a subgame-perfect €/2-equilibrium of the game Q(u), we know that for every p,q € P and every i €I 
if p <; q then u,(q) < u;(p) + €/2. Similarly, to show that o is a subgame-perfect 0-equilibrium of the game Q(v), 
it suffices to prove that for every p,q € P and every i€ I if p<; q then v;(q) < 0,(p). 


Claim 3. The strategy profile o is an SPE of the game Q(v). 


Proof of Claim 3. Fix a player i € I and a play q € P. Let E = {p € P: p <; q}. Notice that ECS. For a play p € E 
let O(p) denote the formula v,(q) < v;(p). We prove that @(p) holds for each p € E by induction on <j. 

Take a play p € E. Suppose that @(r) holds for every r € E with r <, p. We now show that ©(p) also holds. 
Suppose ©(p) is false, that is, v;(p) < v;(q). Since both v,(p) and v;(q) are multiples of e, we have v;(p) < v;(q)—€. 

We have 0;(q) = u; (q). Indeed, if q € P\S, this follows automatically by the definition of v;, and if q € S, this 
follows by (16) since v;(p) < v;(q) implies that v;(p) < u(q). 

Furthermore, 0;(p) = u; (p). To prove the equality, we have to argue that u;(p) < v;(r) for each r € S with r <; p. 
Thus take any such r. We have r € E, and ©(r) implies that v,(q) < v;(r). Since by assumption v,(p) < 0;(q), we 
have v;(p) < v,(r) and consequently u*(p) < v,(r), as desired. 

We obtain the inequalities 


u(p) <u; (p) =0,(p) < 0,(q) -—€ =u; (q)—€ < u,(q) -€, 


contradicting the fact that o is a subgame-perfect €/2-equilibrium of Q(u). o 


This completes the proof of Theorem 8.3. o 
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Endnotes 
'Thus H is a pruned tree and P is the set of branches of H. 


?Suppose first that H is uncountable. Let H, denote the set of histories h € H of length k. Since H = Uxen Hy, there is some k €N such that 
H, is uncountable. Consider the cylinder sets P(A), for h € H,. They are nonempty, mutually disjoint, and there are uncountably many of 
them, so (P,7) is not separable. Now suppose that H is countable. For every history h € H, take an arbitrary play p, € P(h). Then, the set 
{p, |h € H} is countable and dense in (P, 7). 


3Throughout the paper we maintain the following convention: The players of the original game Q are referred to as players with a lower 
case p. The two players of the auxiliary zero-sum games G;(h) are referred to as Player I and Player II with a capital P. 


4Consider P; as a function from H to 2? and take any ordinal €, with cardinality larger than that of the set of such functions. 


5We remark that a much stronger result is proven in Flesch et al.: for small but positive €, the game has no e-SPE, even if one allows for 
mixed strategies. 


®The result is not explicitly stated in the paper. However, most of the proof of Theorem 2.3 is in fact the proof of the existence of an ODI 
strategy profile. 


7More precisely, consider the set E of nonterminal histories h, i.e., histories h for which at least one player’s payoff function is not constant 
on P(h). The usual < relation on E is well founded. Backward induction could therefore be formalized as a recursion on the relation <. 


8For instance consider a game with the action set A =N. Fix an action a* € A and let the function u; be given by u;(a9,4,,...) =1 if Ay =U 
and 0 otherwise. Then u; is continuous but is not continuous at infinity. 
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